Nonstrict hierarchical reinforcement learning for interactive systems and robots

Conversational systems and robots that use reinforcement learning for policy optimization in large domains often face the problem of limited scalability. This problem has been addressed either by using function approximation techniques that estimate the approximate true value function of a policy or by using a hierarchical decomposition of a learning task into subtasks. We present a novel approach for dialogue policy optimization that combines the benefits of both hierarchical control and function approximation and that allows flexible transitions between dialogue subtasks to give human users more control over the dialogue. To this end, each reinforcement learning agent in the hierarchy is extended with a subtask transition function and a dynamic state space to allow flexible switching between subdialogues. In addition, the subtask policies are represented with linear function approximation in order to generalize the decision making to situations unseen in training. Our proposed approach is evaluated in an interactive conversational robot that learns to play quiz games. Experimental results, using simulation and real users, provide evidence that our proposed approach can lead to more flexible (natural) interactions than strict hierarchical control and that it is preferred by human users

Synthesis, Infra-red, Raman, NMR and structural characterization by X-ray Diffraction of [C12H17N2]2CdCl4 and [C6H10N2]2Cd3Cl10 compounds

The synthesis, infra-red, Raman and NMR spectra and crystal structure of 2, 4, 4- trimethyl-4, 5- dihydro-3H-benzo[b] [1, 4] diazepin-1-ium tetrachlorocadmate, [C12H17N2]2CdCl4 and benzene-1,2-diaminium decachlorotricadmate(II) [C6H10N2]2Cd3Cl10 are reported. The [C12H17N2]2CdCl4 compound crystallizes in the triclinic system (P-1 space group) with Z = 2 and the following unit cell dimensions: a = 9.6653(8) angstrom, b = 9.9081(9) angstrom, c = 15.3737(2) angstrom, alpha = 79.486(1)degrees, beta = 88.610(8)degrees and gamma = 77.550(7)degrees. The structure was solved by using 4439 independent reflections down to R value of 0.029. In crystal structure, the tetrachlorocadmiate anion is connected to two organic cations through N-H...Cl hydrogen bonds and Van Der Waals interaction as to build cation-anion-cation cohesion. The [C6H10N2]2Cd3Cl10 crystallizes in the triclinic system (P-1 space group). The unit cell dimensions are a = 6.826 (5)angstrom, b = 9.861 (7)angstrom, c = 10.344 (3)angstrom, alpha = 103.50 (1)degrees, beta = 96.34 (4)degrees and gamma = 109.45 (3)degrees, Z=2. The final R value is 0.053 (Rw=0.128). Its crystal structure consists of organic cations and polymeric chains of [Cd3Cl10]4- anions running along the [011] direction, In The [C6H10N2]2Cd3Cl10 compounds hydrogen bond interactions between the inorganic chains and the organic cations, contribute to the crystal packing. PACS Codes: 61.10.Nz, 61.18.Fs, 78.30.-jComment: 19 pages, 10 figure

Making effective use of healthcare data using data-to-text technology

Healthcare organizations are in a continuous effort to improve health outcomes, reduce costs and enhance patient experience of care. Data is essential to measure and help achieving these improvements in healthcare delivery. Consequently, a data influx from various clinical, financial and operational sources is now overtaking healthcare organizations and their patients. The effective use of this data, however, is a major challenge. Clearly, text is an important medium to make data accessible. Financial reports are produced to assess healthcare organizations on some key performance indicators to steer their healthcare delivery. Similarly, at a clinical level, data on patient status is conveyed by means of textual descriptions to facilitate patient review, shift handover and care transitions. Likewise, patients are informed about data on their health status and treatments via text, in the form of reports or via ehealth platforms by their doctors. Unfortunately, such text is the outcome of a highly labour-intensive process if it is done by healthcare professionals. It is also prone to incompleteness, subjectivity and hard to scale up to different domains, wider audiences and varying communication purposes. Data-to-text is a recent breakthrough technology in artificial intelligence which automatically generates natural language in the form of text or speech from data. This chapter provides a survey of data-to-text technology, with a focus on how it can be deployed in a healthcare setting. It will (1) give an up-to-date synthesis of data-to-text approaches, (2) give a categorized overview of use cases in healthcare, (3) seek to make a strong case for evaluating and implementing data-to-text in a healthcare setting, and (4) highlight recent research challenges.Comment: 27 pages, 2 figures, book chapte

A Biological Global Positioning System: Considerations for Tracking Stem Cell Behaviors in the Whole Body

Many recent research studies have proposed stem cell therapy as a treatment for cancer, spinal cord injuries, brain damage, cardiovascular disease, and other conditions. Some of these experimental therapies have been tested in small animals and, in rare cases, in humans. Medical researchers anticipate extensive clinical applications of stem cell therapy in the future. The lack of basic knowledge concerning basic stem cell biology-survival, migration, differentiation, integration in a real time manner when transplanted into damaged CNS remains an absolute bottleneck for attempt to design stem cell therapies for CNS diseases. A major challenge to the development of clinical applied stem cell therapy in medical practice remains the lack of efficient stem cell tracking methods. As a result, the fate of the vast majority of stem cells transplanted in the human central nervous system (CNS), particularly in the detrimental effects, remains unknown. The paucity of knowledge concerning basic stem cell biology—survival, migration, differentiation, integration in real-time when transplanted into damaged CNS remains a bottleneck in the attempt to design stem cell therapies for CNS diseases. Even though excellent histological techniques remain as the gold standard, no good in vivo techniques are currently available to assess the transplanted graft for migration, differentiation, or survival. To address these issues, herein we propose strategies to investigate the lineage fate determination of derived human embryonic stem cells (hESC) transplanted in vivo into the CNS. Here, we describe a comprehensive biological Global Positioning System (bGPS) to track transplanted stem cells. But, first, we review, four currently used standard methods for tracking stem cells in vivo: magnetic resonance imaging (MRI), bioluminescence imaging (BLI), positron emission tomography (PET) imaging and fluorescence imaging (FLI) with quantum dots. We summarize these modalities and propose criteria that can be employed to rank the practical usefulness for specific applications. Based on the results of this review, we argue that additional qualities are still needed to advance these modalities toward clinical applications. We then discuss an ideal procedure for labeling and tracking stem cells in vivo, finally, we present a novel imaging system based on our experiments

A new benchmark dataset with production methodology for short text semantic similarity algorithms

This research presents a new benchmark dataset for evaluating Short Text Semantic Similarity (STSS) measurement algorithms and the methodology used for its creation. The power of the dataset is evaluated by using it to compare two established algorithms, STASIS and Latent Semantic Analysis. This dataset focuses on measures for use in Conversational Agents; other potential applications include email processing and data mining of social networks. Such applications involve integrating the STSS algorithm in a complex system, but STSS algorithms must be evaluated in their own right and compared with others for their effectiveness before systems integration. Semantic similarity is an artifact of human perception; therefore its evaluation is inherently empirical and requires benchmark datasets derived from human similarity ratings. The new dataset of 64 sentence pairs, STSS-131, has been designed to meet these requirements drawing on a range of resources from traditional grammar to cognitive neuroscience. The human ratings are obtained from a set of trials using new and improved experimental methods, with validated measures and statistics. The results illustrate the increased challenge and the potential longevity of the STSS-131 dataset as the Gold Standard for future STSS algorithm evaluation. © 2013 ACM 1550-4875/2013/12-ART17 15.00

Technology options for aged care in Japan

Demographic changes after the Second World War have resulted in the Japanese people having the world\u27s highest life expectancy. This, coupled with the declining birth rate, has caused a rapid growth in the proportion of aged people in the population and a declining proportion of the population in the workforce to support the non-working population. At the same time, post-War changes in social conditioning and family structures are diminishing the availability and reliability of family members to act as caregivers to the aged. Historical evidence suggests that the Japanese people, and in particular the government, are open to finding technological solutions to address their social and economic problems. One method of dealing with the scarce resources for aged care in Japan is to develop and utilise technology that assists the elderly to maintain as much independence for as long as possible and to assist care-givers by easing their workload. This thesis examines three technology options that can be used to help aged care in Japan today. Option One uses only standard technology from other countries; Option Two calls for investment primarily in high-level technology development, exemplified by robotics technology; Option Three concentrates on developing and using smaller-scale technology such as barrier-free technology for aged care. Details of the development of robotics and barrier-free technologies and the assumptions underlying their development for aged care are examined. The thesis discusses ways in which these options affect various stakeholders—the government, researchers, professional care-givers, family care-givers and recipients of the care. This framework for assessing technology for aged care—spelling out options, unearthing assumptions underlying the options and surveying effects—is a useful and convenient tool for policy makers and other interested parties

Japanese technology policy for aged care

Aged care provision is a burning issue in Japan due to the country\u27s unparalleled longevity and a fraying tradition of children caring for parents. Use of technology offers one approach to helping ease the increasing burden of aged care. Ways of using technology can be conveniently classified into three options. The first is to rely on well-tested technologies developed outside Japan. The second option is for significant Japanese investment in high-technology aged care supports, including robotics. The third option is for significant Japanese development in barrier-free technology, a low-technology direction. Articulating these options and spelling out their likely consequences helps to highlight some of the implicit value judgements involved in Japanese technology policy for aged care